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ABSTRACT 


A recent audit by the Department of Defense Inspector General (DODIG) of Navy 
inventory control points found a high value of ]»irchase requests for repairable hems that 
the auditors labeled as unnecessary or excessive. The dollar figures rqrorted were based 
on the auditors’ use of stratified sampling. This thesis examined the audhors’ use of 
stratified sampling by attempting to replicate the audhors’ process of stratifying and 
sampling. The author then attempted to verify the audhors’ claimed confidence level and 
precision of the final resuh. This study questions the chosen sample and sample 
stratification. In addition, this thesis found that the audhors’ actual precision was not as 
tight as stated in the DODIG audit report. This was caused by the audhors’ emphasis on 
the very high dollar value strata which had only a few purchase requests rather than on the 
stratum with the largest number of purchase requests. It was this latter stratum which had 
the highest projected number and total dollar value of excesses. 
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L INTRODUCTION 


A. MOTIVATION 

This study is based on the results of an economy and efiSdency audit conducted by 
the Department of Defoise Inspector General (DODIG) from August 1990 throu^ 
November 1991 in accordance with standards issued by the Comptrollo’ General of the 
United States (Jones, Fd}ruary 1,1993, p. 2). The DODIG conducts audits of the Navy's 
inventory control points’ (ICP) procurement actions using a multi-stage sampling plan that 
incorporates stratified sampling. Using these stratified samples, DODIG conducts audit 
tests to determine which purchase actions are for "reasonable" quantities. 

When the results of this audit were reported, they also identified a total dollar 
value of items that were considered unnecessary or premature purchases. In this case, the 
DODIG identified potential monetary benefits totaling S71.7 million, which represents the 
estimated value of unnecessary purchases. When audit findings disclose such "problem" 
purchases, obligational authority is reduced for the ICPs. The reasoning is that, if the 
funding had not been wasted on these particular procurement actions, the funding would 
not have been needed at all. In order to avoid inaccurate conclusions, it is absolutely 
critical that such audit findings accurately refiect weaknesses in invemory management. 

B. OBJECTIVE 

The focus of the thesis will be to look at how the DODIG incorporates stratified 
sampling methodologies in auditing major procurement actions and then making recom¬ 
mendations to inventory control points. In particular, this thesis will look at how the 
DODIG gathers the audit results from each stratum sampled and projects these findings 
over the entire population of procurement actions. This study explores the question of 
whether sampling stratification with current sample sizes can be used to make projections 
of potential monetary savings with a high confidence level and tight precision. 
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C. RESEARCH QUESTIONS 

The primary question to be answered in this project is: "Does the DODIG's 
statistical sampling plan and stratified sampling of procurement actions lead to the most 
accurate conclusions and recommendations?" In order to answer this question compl^dy, 
secondary questions concerning choices of stratifications, size of strata, and confidence 
levels within each stratum must be addressed. In particular, how is tlus sampling structure 
used to project errors in each stratum to the population as a whole? 

D. SCOPE, LIMITATIONS, AND ASSUMPTIONS 

The scope of this study is very narrovdy defined in order to remain focused on the 
research questions mentioned above. In the audit report analyzed, there are many areas of 
debate that are clearly not resolved. Each one could be the basis for a separate study at a 
later date. This project will look at the baas for the amount of potential monetary 
benefits suggested in the audit report. As the report's Executive Summary states, the 
Navy disagrees with the basis for estimating potential monetary benefits (Jones, February 
1, 1993, p. u). 

This project's primary focus is on how the DODIG used sampling to make its 
observations of the ICPs' procurement of repairable items. For example, this theas 
considers the initial total quantity of line items, the snudler universe of line items chosen to 
sample, the adjusted universe of line items to sample, the sample of procurement actions 
to be scrutinized and the conclusions drawn fi-om these samples. The area analyzed 
concerns the dollar value projection based on these findings. In statistical terminology, 
this thesis primarily focuses on the dollar value estimates of the urmecessary purchases 
reported in the audit to see if improvements could have been made and, secondarily, 
examines the fi'equency estimation. Dollar value and firequency estimation are defined in 
Chapter II, Section B. 
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This study is not intended to settle disagreements on terminology, determine 
whether the audit was poorly timed, nor make reconunendations on how internal controls 
should be changed. Instead, this thesis concentrates on the statistical processes and on 
how they resulted, dther accuratdy or inaccurately, in budgetary recommendations for the 
ICFs under review. 

E. PREVIEW 

The remaining chapters are organized in the following manner. Ch^ter II gives 
background information on stratified sampling, estimation sampling, confidence intervals, 
and the inventory management definition of stratification. 

Chapter HI presents all the data provided by the statisticians at the DODIG. The 
data is broken down as described in the audit report, and it shows the conclusions derived 
firom this data. The chapter concludes with an oq}laiution of how the audit results were 
actually used by the affected command when submitting a budget request 

Ciuq)ter IV opens with a discussion of how the DODIG made its statistical 
decisions. The following section begins the author’s analyses of the statistical decisions. 
The first analysis is of the selected universe size and sample size, followed by an 
alternative method to choose a sample size, and, finaUy, how to stratify the chosen sample 
size. The second major analysis is of the auditors’ stated confidence level and related 
precision. The chapter closes with a comment on the several numerical discrepancies 
found in the audit report and worldng papers. 

Chapter V summarizes the main points and conclusions of the thesis and presents 
several recommendations. 
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n. METHODOLOGY 


A. STRATIFIED SAMPLING 

1. Stnittficd Sampling Defined 

In stratified random sampling, the population is divided imo a smies of 
independoit subpopulations which are called "strata." Each stratum is sampled as though 
it were a separate population from which an unrestricted random sample were being 
drawn. It is then posable to combine the separate results to make a conclusion about the 
entire population. A degree of predaon can then be applied to these results (Cyert, 1962, 

p. 116). 

2 . Why stratified sampling? 

Auditors may not consider all accounts or records of equal inq>ortance. For 
instance, they may have a much greater interest in establishing the accuracy of large 
accounts and be unwilling to run as great a sampling risk for this type of account. 
Objective sampling methods do not necessarily require sampling fix}m a general pool of 
items. It is not only posable but often desirable to s^regate the population into separate 
groups, by size or other characteristics, and to sample vdth various degrees of accuracy in 
each area separately. 

When sampling accounts, it is common to want to examine all of the large 
accounts, a large portion of the moderately azed accounts, and a relatively small 
proportion of the lowest doUar-valued accounts. In order to draw a clear concluaon fix)m 
this process, it is necessary to stratify the accounts by size and then analyze the results. 
This procedure is looked upon favorably because there is improved sample reliability for 
the large accounts. Because it removes the large accounts fi'om other sections of the 
sample, determination of the sampling variability also is more precise (Hill, 1962, p. 45). 
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3. How to Use Stnitined Sampling 

With stratified sampling of audit units, the heterogeneous population (Figure 
1 A) is first divided into a nund>er of mutually excluave groups or strata (I .B). After the 
population of audit units has been divided into a number of mutually exclusive strata, a 
simple random sample (without replacement) of purchases is selected from each stratum 
(1 .C). These samples must be independent, which can be accomplished by use of different 
sets of random numbers for the selection of the different random samples. After the 
samples are pulled fi’om the strata, they are analyzed for specific attributes or values (1 .D). 
Finally, based on the resuhs of the analysis, a concluaon is drawn about the original 
population(l.E) (Neter, 1975, p. 77). 


(A) Heterogeneous 
population 


(B) Heterogeneous 

population divided into 
homogenous strata 


(C) Samples are pulled 
from strata 


(D) Samples analyzed for 
specific attributes 


(E) Based on results from 
(D), projections are 
made about 
population (A) _ 



.\ ^ ^ 
Cja^pl* d^Ta> 



Figure 1. Stratified Sampling Process 
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4. Advantages of stratified sampUng 

There are three major advantages of using stratified sampling: 

(1) Efficiency . Frequently a stratified sample can produce a satisfiutory result with a 
minimum of effort and expense when conq>ared to a simple random san^le. One 
probability sampling procedure is more ^dent than another if it offers, at a given level of 
confidence, the same predsion at less cost or greater precision at the same cost. For 
example, a hypothetical sample of nine teachers could be observed to find their average 
years of education. In Table I their variance equals 2.61. However, when teachers are 
stratified by the levd of school t* '' teach and the two teachers with the greatest 
difference in years of education are sampled, the variance of each stratum significantly 
declines. These variances, which measure the variability of years of education within each 
stratum, are much smaller than the variance of the entire population. Consequently, in this 
instance, a stratified random sample of a givoi aze will yield a substantially more precise 
estimate than a simple random sample of greater size, since much smaller variability is 
encountered within each stratum. 


Table L THE EFFECTS OF STRATMCATION 


Teacher 

Level of School 

Years of 
Education 

1 

Teacher 

Level of School 

Years of 
Education 

1 


13.5 


6 


20.5 

2 


19.0 


7 


17.5 

3 


16.5 

1 

8 

wmsiiwm 

19.5 

4 

Elementary 

16.0 

■ 

9 

Elementary 

16.5 

5 


18.0 





Variance = 2.61 

Stratum 1; Elementary 

Stratum 2; High School 

Stratum 3; College 

Teacher 

Years of 
Education 

Teacher 

Years of 
Education 

Teacher 

Years of 
Education 

1 

15.5 

3 

16.5 

2 

19.0 

9 

16.5 

5 

18.0 

6 

20.5 

Variance = 0.25 

Variance = 0.56 

Variance = 0.56 

Source: Applied Statistics, Neter, 1993, p.735 
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However, for stratified sampling to be efficient, the strata must be designed to 
contain relatively homogeneous elements. Homogeneity is accomplished when the basis 
of stratification is related to the charactoistic under study (e.g., level of school taught and 
years of education). While the process of stratifying a sample may be considered an extra 
effort when compared to taking a simple random sample of equal total size, the 
stratification process normally allows for a decreax in the total sample size (see Table I). 
This decrease in total sample size is desirable because precision can be maintained, or 
improved, and the sampling time and costs are reduced. 

(2) Information about subpopulations . Stratified sampling can provide secondary findings 
of strata characteristics in addition to the primary findings of overall population 
characteristics. For example, a study by a university of the effects of tuition increases may 
be primarily meant to provide the extent of sentiment fi’om all students, but it also 
provides valuable separate information about graduate and undergraduate students, 
minority students, male and female students, etc. (Neter, 1993, p. 735). 

(3) Feasibility . Sometimes stratified sampling is simply the most feasible. For example, if 
police records are computerized in a city and manual in the rest of the state, administrative 
consideration may require separate sampling in the city and elsewhere in the state (Neter, 
1993, p. 735). 

B. ESTIMATION SAMPLING 

The fimction of estimation sampling for auditors is to determine two things. First, 
an attempt is made to determine the number of occurrences of some attribute, such as 
errors, violations, etc. in the population. Secondly, once the attributes are identified, it is 
usually appropriate to determine the m^nitude of those attributes in order to make 
suggestions for improving the operation imder review. 
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1. Frequency Estinuition (Attributes) 

Often it is sufiBcient simply to determine how many times a certain attribute 
occurs. Once this has been found it is only necessary to total the occurrences beft>re 
making conclusions and judgments about the situation that was observed. This is 
particularly true when all attributes (errors, violations, etc.) carry equal weight and one is 
not more important than anotha*. 

During the DODIG audit, the "attributes” being checked were the occurrence 
of excessive (unnecessary or premature) purchases. Because each one of the excessive 
purchases observed had a difierent dollar value, they were not sampled equally. Instead, a 
higher percentage of the purchase requests in the Ugh dollar value strata were selected. 

2. Dollar Value Estimation (Variables) 

Variables estimation is the statistical method used to estimate dollar values. The 
objective of this method is to estimate an average value of a group of items by means of a 
sample with an assurance equivalent to the confidence level that the sample average will 
be within a range of some specified amount fix>m the true average which would have been 
attained if all items in the entire population had been averaged (Hill, 1962, p. 31). 

The distinction between fi'equency and dollar value estimation can be seen when 
examining invmitories. On the one hand, a company may want to check the reliability of 
inventory control. Items in the storeroom are identified and then compared to the 
invemory records to ensure that a stock record exists for every item. On the other hand, 
the compare may want to determine the value of its inventory. A sample of the inventory 
would be counted, priced, and extended. On the basis of the average value of the items 
selected, the aggregate value of the inventory could be determined within the limits of 
reliability attained and compared to the compai^r’s figures (Hill, 1962, p. IS). 

In the case of the DODIG and the ICPs, the sample purchase requests were 
analyzed and a determination was made as to whether or not they were excessive. 
Secondly, the DODIG determined from a relatively small sample of purchase requests the 
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dollar value of all excessive purchase requests. This mettod of ddlar value estimation will 
be examined more closely in Chapters in and IV. 

C. CONFIDENCE INTERVALS 

In the DODIG report, the estimated cost savings ate given with a 95 percent 
confidence level and a sampling precision of ± 3% (Jones, February 1, 1993, p. 33). 
Therefore it is critical to know exactly what this means. Confidence intervals are used for 
interval estimation. Interval estimation is the quoting of bounds between vtdiich the true 
population mean lies. This is appropriate when it is deared to 0 ve some indication of the 
accuracy with which a parameter is estimated by the sample (Rolnnson, 1992, p. 120). 

In any situation udiere there is less than a conq>lete sample (e.g., less than the 
entire population), there ^ be a mar:^ of uncertainty surrounding the results. This 
degree of uncertainty may be measured mathematically and expressed in terms of 
reliability (e.g., 95% confidence) and precision (e.g., ± 3%). Precision and reliability are 
mathematically interdependent and statistically inseparable. “The precision of an estimate 
indicates the range within which it is expected to be accurate, and the reliability (or 
confidence) is the probability of achieving this accuracy” (Newman, 1976, p. 30). 

Probability statements caimot be made about a single event. For example, if a coin 
is flipped once, it cannot be said that the probability of the coin having turned up heads is 
one-half The coin ather turns up heads or it does not. I^ however, the coin is flipped 
numerous times, a meaningful statement can be made that the relative firequency of heads 
turning up approaches 0.5 (Cyert, 1962, p. 8 ). 

The reliability statements made about a confidence interval are similar to the 
statements on coin flipping. It is inaccurate to specify the probability that a particular 
confidence interval contains the true value being estimated. However, meaningful 
statements can be made about the probability that similar confidence intervals, in a series 
of sample estimates, contain the true value being estimated. For e iple, it might be 
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appropriate to make the statement that, in repeated trials, a confidence iitterval of |rfus or 
minus “x" dollars about a sample estimate will contain the true value being estimated SK)% 
of the time (Cyert, 1962, p. 8). 

An important question is iKdun aze confidence interval should be used? 
answer depends on the tradeoff between the confidence levd and the widt he interval. 
To obtain higher confidence fix)m the same sample, one must be willing to du:ept a larger 
margin of error (wider interval). The way to attain higher confidence and still have a short 
interval is to take a larger sample. That means that, for a fixed level of confidence (e.g., 
95%), the confidence interval (e.g., ± 3%) decreases as the sample size increases. Jr, for 
a give interval, the confidence level inoeases as the sample size increases (Moore, 1979, 
p. 276). 

D. STRATIFICATION 

The final bottom line impact of this audit conned of the values for three 
measures • the universe of contracts, the percoit of unnecessary contracts, and the Due In 
Long Supply (OILS) rate. Each of these is fiuther discussed in Chapter m.F. However, 
because the DILS rate is determined through the use of “stratification,” this term, as it 
applies to the DILS rate, must first be clarified. 

It is important to understand that we are talking about the inventory management 
definition of stratification (also called “STRAT”) which is an entirely separate concept 
fi'om the stratified sampling process de«nibed earlier. A complete description of 
stratification and all its related equations is beyond the scope of this thesis. However, it 
should be understood that stratification is mainly the semi-aimual process of comparing 
various assets and requirements levels and placing each in a priority sequence. The 
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stratification program is used to analyze each inventory hem. After all items have been 
stratified, the total stratification budget request is (ri>tained by adding up the dollar value 
of future expected procuremoit/rqNur defidendes during the Budget Year (FMSO 
S230/S2A, July 17, 1994, p. 1-1). Stratification is the basis for budget requests needed to 
operate the inventory system. 

Because stratification tries to predict future needs, h uses three periods over which 
nuiterial usage must be considered; Current Year, ^portionment Year, and Budget Year. 
To initiate the forecast, “STRAT* first looks at the Opening Portion (the “right nov/’ 
inventory and requirements). The Openi' r Position is computed by taking the current 
system assets on hand and matching them against curroit system requirements such as 
safety levels and lead time demand. Next, demands and forecasted order recdpts are 
computed for the Current Year, which covers the remainder of the fiscal year afiier the 
Opening Position date. The ^portionment Year includes the twelve months following 
the Current Year and its demands and forecasted order receipts are likewise estimated. 
Finally, the Budget Year’s demands and required orders are determined. The Budget 
Year extends fi-om the end of the Apportionment Year through to the following 
September 30th. When stratification matches assets against requirements, it may not 
compute material requirements but rather excess inventory and may therefore recommend 
review of long supply conditions, where on hand and due-in material exceeds material 
retention ceilings. 
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m. PRESENTATION OF DATA 


A. INTRODUCTION 

The otyecdves of the DODIG unfit were to detemune \^iether quantities of 
rq>airalfie items to be bou^ on outstanding {Mxxsirements by the Navy v^lesale ICPs 
were warranted by antidpated requirements and wh^her internal management controls 
over the determination of the procurement requirements were effisctive (Jones, February 1, 
1993, p.l). 

>^^thin the last five years, the Dqrartment of Defense Inqsector Genoal has 
changed the type of inventory audits conducted in the services. Previously the audits 
concentrated primarily on Hems that were newly authorized to be placed in inventory. 
Items that were already being carried in stock were not included in the audit. 
Additionally, procurement lead times arrd customer demand were the main areas reviewed 
(Interview with Mr. J. Chaney, April 1994). 

In 1990, DODIG decided to take a slightly dififerent ^rproach to their audit. At 
that time they began to look at ail herns with procurements in process, whether or not the 
procurements were for new hems. Initially, the audits were to be DOD-wide. However, 
DODIG eventually chose to audh service by service and then made a further splh between 
repairables and consumables. 

As of September 30, 1990, the ICPs were in the process of procuring 
approximately $863 million of stock for 11,308 depot level rqiairable (DLR) line items 
(see Colunm (2) in Table U below). The process to purchase these items starts when 
automated programs are triggered to reordo* and the program recommends a purchase 
quantity to meet the stockage objective. The inventory manager then reviews the 
requirement and other relevant data to verify the accuracy of the computation. Approved 
purchase requests serve as authorization for the ICP’s procurement organization to 
the material. 
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Table IL SELECTION OF SAMPLE UNIVB 

RSE 

(1) 

(2) 

(3) 


UNIVERSE 

PURCHASE REQUESTS 
GREATER THAN $100,000 

ASO 


889 

SPCC 


541 

TOTAL LINE ITEMS 

HDim 

1430 







ASO 


$495,900,000 

SPCC 


$195,300,000 

TOTAL DOLLAR VALUE 


$691,200,000 





B. AUDIT SCOPE 

DODIG initiaUy intended to narrow its sample universe from 11,308 to only those 
procurements valued at $100,000 or greater (Column (3) of Table II) independent of ICP. 
This cutoff of $100,000 was chosen by DODIG because purchases of this value required 
greater supervisory review and checking the effectiveness of the review process was one 
purpose of the audit. At the same time, a great majority of the total doUar value of 
procurements in process were included above the $100,000 level. Specifically, this 
decision narrowed the population to 1430 line items (13 percent of 11,308) but stiU 
included 80 percent of the total value of the procurements in process ($691.2 million). 


C. AUDIT SAMPLE 

From the universe of 1430 line itons, initially a sample of 107 line items was 
selected. The 107 line items totaled $229.4 million in procurements (see Column (3) of 
Table III). Appendix B ^ves a detailed summary of all purchase requests greater than 
$100,000. Before the sample was taken, the purchases were identified by ICP, and within 
ICPs they were further subdivided by size of procurement. The four procurement strata 
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were: (1) greater than $5 million, (2) $2.S « S million, (3) SI - 2.49 miilioii, and (4) $100 - 
999 thousand. 


Table m. INITIAL SAMPLE SELECTION 


mnsm 

(2) 

(3) 

(4) 


PURCHASE 
REQUESTS ^lEATER 
THAN $100,000 

SAMPLE FROM 
1430 ITEMS 

ITEMS NOT 
QUALIFIED 
(DISCARI^) 

ASO 

889 

69 

IS 

SPCC 

Ml 

38 

6 

TOTAL 

1430 

107 

21 



(7.5% of 1430) 






ASO 




SPCC 




TOTAL 

$691,200,000 


$41,100,000 






Of the 107 herns (listed in ^pendix C), 21 did not "meet the criteria" of the 
DODIG review. These 21 items (see Column (4) of Table m) were excluded from further 
review dther because the purchases wm not in process at the sanq}le cutoff date (e.g., 
purchase requests were canceled or contracts had already been awarded) or because the 
purchases were for items that were managed uring "consumable item" management 
techniques. It was necessary for audited items to be in process so that ai^ unnecessary 
procurements identified by the audit could be cancded, if the Navy concurred with the 
findings. In reality, SS5.1 million in purchase requests were canceled by the ICPs during 
the audit. Of the SSS. 1 million, $33.0 million were initiated by the ICPs and the hardware 
systems commands (HSC), independent of the audit, and the renuuning $22.1 million were 
canceled in response to the audit (Jones, Fd^ruary 1, 1993, p. 2S). Purchases that were 
excluded because they were treated as consumable items were included in a separate audit 
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aidtled “MUitaiy Department Requirements for Currently Procured Wholesale Inventorfos 
for Consumable Items,” DODIG Report No. 91-106 of June 28, 1991. 

The S41.1 million item adjustment shown in Column (3) of Table IV is segmented 
by ICP and stratum in .A 4 )pen(Ux D. This adjustrnem reduced the final audit samole to 86 
line hems involving purchases valued at S188.3 million. This adjusted sample u anown in 
Column (4) of Table IV and is shown in detail in ^pendbc E. 


Table IV. REVISED SAMPLE SELECTION 


mmsm 

(2) 

(3) 

(4) 


SAMPLE FROM 
1430 ITEMS 

ITEMS NOT QUALIFIED 
(DISCARDED) 

SAMPLED ITEMS 
FOR REVIEW 

ASO 69 

15 

54 

SPCC 

6 

21 

TOTAL 

107 

21 

86 




(80.4% OF 107) 





ASO 

wmssmusssm 

$30,700,000 

$141,900,000 

SPCC 




TOTAL 

jmMXSSSSSWM 

$41,100,000 

$188,300,000 




(82.1%of$229.4M) 


Based on the previous adjustments, the Navy sample universe was adjusted to 
1056 line items with purchases valued at $520.2 million. The theory behind this process is 
discussed in Chapter IV.E. The adjustment of the sample universe was done in two parts - 
by line items and by dollar value. First, the number of line items of the new sample were 
compared to the old to get a percentage value. This value was then multiplied by the size 
of the old universe This was done by stratum for each ICP and they were totaled to give 
the number of line items for the adjusted universe. 
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Table V ASO LINE ITEM ADJUSTMENT 


Stratum 



Old Universe 

New Universe 

1 

8 

7 

8 

7 

2 

20 

15 

26 

20 




IH 



4 

24 

17 

789 

559 


69 

54 

889 

644 


To show one example. Table V has been pulled from Appendix B and ^pendix E. 
The calculation of the adjustment to ASO stratum 3 will be described. First, the sample 
size has changed from seventeen to fifteen. Fifteen divided by seventeen equals .88235. 
When .88235 is multiplied by the old univo^ aze (66), the new universe size is obtained 
for ASO stratum 3; namely 58. This method holds true for each ICP’s stratum and finally 
gives the “adjusted universe” size of 1056. Secondly, the adjustment to obtain the “new 
universe” value is calculated by getting the dollar ^ of the itons discarded from the 
sample for each stratum, dividing it by the origiiud sample value and then multiplying the 
quotient by the old universe value. As an example, in Table VI, stratum 3 is calculated by 
dividing $2,601,156 by $24,738,320. The quotient (.1051) is then multiplied by 
$99,705,919 to get the estimated reduction in the utuverse value of $10,483,762. This 
estimated reduction is labeled the “projected value” in Table VI as that is the term used by 
the DODIG. The final universe value is determined by doing this for each ICP’s stratum 
to get the total value for the entire adjustment (items deleted). That value is 
$170,972,701. When this adjustment value is subtracted from the original universe size, 
the adjusted universe value of $520.2 million is obtained. 


Table VI. ASO VALUE ADJUSTMENT 


Stratum 

Adiustment Value 

Old Sample Value 

Old Universe Value 

Prpjeaed Value 

1 

$9,006,800 

$78,092,994 


■lESBEai 

2 

17,394,859 

64,160,301 

mmmkm 

22,737,337 







99.705»l 

s. 


SB 

4 

1.717.478 

5.648.744 

237.191.420 

72.117.091 


$30,720,293 


$498,856,170 

$114,344,990 
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D. DODIG AUDIT 

DODIG auditors examined the 86 purchase request documents related to the 
purchases in process as of Septonber 30, 1990 to evaluate the basis for procurement 
decisions. They evaluated requirements data that were in effect at the time of the audit to 
determine whether requirements supported continuation of the procuremoit. 

Specifically, the auditors (1) determined whether requirements forecasts were 
reasonable, (2) reviewed the accuracy of the forecasted denumd rates, (3) evaluated tl^ 
propriety of nondemand-based (additive) requirements, and (4) verified the accuracy of 
on-hand asset and due-in asset balances. Additionally, the auditors selectivdy reviewed 
other requirements data and other fiictors that affected the requirements forecast, such as 
past and future program data, survival and wearout rates, and rq>air cycle times (Jones, 
Febriiary 1, 1993, p. 2). The auditors did not n^ely consider whether the purchases were 
justified at the time of reorder, but also v«diether the initial order quantity was still justified 
at the time of the audit in September 1990. Any demand changes after the initiation of the 
audit were not considered part of the audit results. Any action, or inaction, on the part of 
inventory managers was not included in the audit results (Interview with Mr. J. Chaney, 
May 1994). 

E. DATA CATEGORIZATION 

All data presented in this section ultimately lead to the DODIG’s calculation of 
projected monetary benefits (fi'om the canceling of outstanding orders) derived fi’om this 
audit. After all calculations and negotiations, this was determined to be S68.2 million. 
Reading Table VII fi’om left to right shows the progression to reach the projected 
monetary benefits (listed as “PMB”). First, the universe is divided into three categories. 
Second, the excessive purchase requests are pulled fi’om the universe and divided into 
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three categories. Third, unnecessary purchase requests are pull«i fixrm the excessive 
purchase requests and divicted into two categories. Fiiudly, the (mrchase requests with a 
“potential monetary benefit” are pulled fi’om tlM unnecessary purchases to detamirre tlw 
findings of the audit. The “other” category represents ofl&etting costs that reduce net 
potential benefits (Jones, February 1, 1993, p. 47). Each one of the eight groupings, 
except “Other,” is broken down by ICP and stratum in Appendices G through M. 


Table m SUMMARY OF AUDIT RESULTS 


_Q)_ 

(2) 

(3) 

Adjusted 

Universe 

Excessive 

Unnecessary 

Undetermin. 

$69,543,164 


$46,046,289 


$6$JS45w88T 

Reasonable 


Premature 

mmmm 

Other 

10.507.569 














The first step after the doUar value of the adjusted universe was determined was to 
divide this universe dollar value imo three non-overiapping categories • undeterminable, 
reasonable, and excessive. The total combined dollar value of these three categories must 
equal the dollar value of the entire universe ($523.2 million). This total differs fi'om the 
previous total value for “adjusted universe” ($520.2 million) described in Section C and 
the reason for the difference will be discussed in Chapter IV.E. 

1. Undeterminable 

The first category into which the universe was subdivided was called “unde¬ 
terminable.” Undeterminable included all purchase requests for which DODIG was unable 
to determine the reasonableness of purchases because the ICPs could not provide 
verifiable requirements data as of September 30, 1990 or the requirements were dependent 
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on a management decision by an HSC (Jones, February 1, 1993, p. 34). Purchase requests 
were considered “eccessive” if the quantity ordered exceeded the DODIG-defined 
“stockage objective” by more than twdve months of forecast requiranoits. Purchase 
requests that did not fit into either of the these categories were consid«^ “reasonable.” 

The method to make the projection based on the sample is id«itical for each of 
these three categories. First, the quantity (hits) and dollar values (hit values) are 
determined. A purchase request is considered a “hit” if it either wholly or partially fits 
into one of the three categories. For instance, one purchase request may be partially 
reasonable and partially excess. However, the dollar value of the two groupings cannot 
exceed the original purchase request value. For instance, a purchase request for $110,000 
may be determined to be $10,000 excess by the auditors. This one line item would then fit 
the “reasonable” category with a “hit” value of $100,000 and the “excessive” category 
with a “hit” value of $10,000. Although the term “hit” implies a problem, the DODIG 
uses the term simply to classify purchase requests and their dollar values. 


Table VIIL ASO 

UNDETERMINABLE LINE ITEM PROJECTION 

Stratum 

Hits 


Universe Quantity 

il£S55gil.{ia 

1 

2 

8 

8 

2 

2 

4 

20 

26 

5 


3 


66 

12 

4 

1 

24 

789 

M 


11 

69 

889 

85 


To make the projection, the hit quantity is divided by the sample quantity and 
that quotient is then multiplied by the universe quantity. The answer is then rounded to 
the nearest integer. Excerpts from Appendices B and G are given as examples. In Table 


19 



























VIII, ASO stratum 3 shows three undeterminable items. Three is then divided by the 
sample quantity of seventeen. The quotient (.17647) is then multiplied by the universe 
quantity of 66 to get twelve projected hits for stratum 3. The same methodology is used 
to get the projected values in Table DC. Specifically, $4,646,285 divided by $24,738,320 
equals .187817321. Then the universe value of $99,705,919 multiplied by .187817321 
equals $18,726,500 (the projected value for ASO stratum 3). 


Table DC. ASO UNDETER] 

MINABLE VALU 

E PROJECTION 

mm 

(2) 

_ m _ 

(4) 

(5) 

Stratum 

Hit Value 


Universe Value 


1 





2 



mmmm 







4 

389.754 

5.648.744 

237.191.420 

16.365.837 


wmmsm 

$172,640,358 

$498,856,170 

$69,543,164 


A summary of the projected hits and projected values for undeterminable, 
reasonable, and excessive purchase requests is shown in ^pendices G through I. The 
total projected values of these three categories does equal $523,168,692 as shown in 
Table VH. 

2. Excessive 

The next major step the DODIG took to determine projected monetary benefits 
was to break down the “excessive” category into three subcategories. The DODIG calls 
these groups non-projectable, premature, and urmecessary. The total of these three areas 
should equal $156,563,299, wluch is the total for the “excessive” category as shown in 
Appendix I and Column (1) of Table VII. However, a thorough review of all working 
papers shows that these three actually total $156,923,374 - a difference of $360,075 
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(Barton, Nov. 16,1992, summary page). The impact of this difference will be discussed in 
Chapter IV.E. 

The method used to project the number of hits and the hit value in the univo^ 
was the same as that used to make projections for the adjusted universe described in the 
previous section. The number of hits in the sample stratum are divided by the stratum 
sample size and the quotient is multiplied by the universe size. The product is then 
rounded to the nearest whole number. This is then done for each stratum and the results 
are totaled for each ICP to get the projected hits and projected total hit value. 

3. Unnecessary 

The final category that the DODIG calculated was the “projected monetary 
benefit” portion of the purchase requests determined to be unnecessary. The DODIG 
projected $68.2 million of potential monetary benefits fi’om the unnecessary items (see 
Table VII). The $10.6 million difference ($78.8 million total uimecessary minus $68.2 
million) represents offsetting costs to repur unserviceable assets (Jones, Fd)ruary 1, 1993, 
p. 47). However, the audit report does not explain the specific nature of those costs or 
how or why they ofiset benefits fi’om avoiding unnecessary purchases. The proje on of 
the potential monetary benefits can be seen in Appendbc M. The methodology used to 
make the projection is also identical to the calculations used to nuike projections for the 
adjusted universe as described above in section E. 1. 

F. AUDITIMPACT 

The actual firud impact of this audit was reported in an NC-2 Report submitted by 
the Naval Supply Systems Command (NAVSUP) with the reapportionment budget to the 
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Navy Comptrollo' (NAVCOMPT). This report summarized all audit findings and 
recommendations fi'om the previous fiscal year. The purpose of this report was to ensure 
that monetary benefit claims (iHidget reduction) are actually incmporated into the 
following year’s budget. 

The bottom line for this particular DODIG audit was calculated using three 
measures: 

(1) 12 (universe) - $520.1 million. The value of the adjusted universe size reported in the 

audit. 

(2) Q (unnecessary contracts) - 71%. The audit projected SI 10.9 million in premature or 

unnecessary procurements (see Table VII). Unnecessary contracts were projected 
to total $78.8 million (or 71% of $110.9 million). 

(3) D (DELS) - 8.23%. The Due In Long Supply (DILS) rate calculated during each 

semiannual STRAT (as described in Chapter n.D). NAVSUP figured this DILS 
rate by estimating the total value of all contracts in excess of known requirements 
during the Budget Year and dividing this figure by the total value of contracts 
expected to be on order during the Budget Year. The average DILS rate firom 
September 1990 through September 1991 was 8.23%. 

To obtain the figure that NAVSUP used to report to NAVCOMPT, the following 
equation was used: U x C x D = Agreed Potential Monetary Benefit. NAVSUP reported 
this agreed potential monetary benefit to be $30.4 million or $520. IM x 71% x 8.23%. 
This figure was used by NAVSUP in their arbitration hearing with the DODIG. The 
arbitrator was a representative from the Office of the Undersecretary of Defense for 
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Logistics (OUSD(L)). OUSD(L) determined that $30.4 million was accq>table for 
NAVSUP to report to NAVCOMPT. Furthermore, because the $30.4 million was 
classified as a one time “cost avoidance” for fiscal year 1991 rather than “savings” fi'om 
the audit, NAVSUP was not required to incorporate this figure into their budget estimate 
for fiscal year 1994 but rather simply to list it as a recognized audit finding. In other 
words, a cost would have occurred if the audit had not taken place. However, because 
there was an audit, the costs are said to have never occurred. 
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IV. DATA ANALYSIS AND INTERPRETATION 


A. INTRODUCTION 

This clu 4 )ter indudes a critique of the entire (nocess used by the DODIG to make 
conclusions found in the audit report. Specifically, Section B discusses the DODIG 
selection of the universe size and sample size, strata, confidence levd, and precision. This 
information was obtained fix)m their audit report, the audit working papers provided by 
the Quantitative Methods Division of the DODIG, and discussion with the statisticians 
assigned to the audit. Section C begins tlw analysis of the DODIG dedaons and then 
presents an alternative method of sdecting the sample size along with two ways of 
dividing the sdected sample over the chosen strata. Section D provides a method to test 
stated levels of confidence and precision. The chapter concludes with a discussion of 
several incondstencies in the reported figures used during the audit. 

B. DODIG STATISTICAL DECISIONS 

1. Adjusted Universe 

As mentioned in Chapter in.C, the DODIG treated hems that they did not want 
to include in the audit in the same way as the attributes they did want to include. For 
example, when their samples uncovered excessive or reasonable purchase requ^s, 
projections were used to calculate the number and value of these purchase requests in the 
universe. At the same time, when the DODIG found sampled purchase requests that they 
determined were outside the scope of the audit, they used the same projection procedures 
to “adjust” the universe. While the process would have been time consuming, the only 
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statistically valid way to ‘‘adjust” the universe is by going through the original universe and 
discarding the purchase requests deemed to be outside the scope of the audit. Only after 
this is complete can a true sample be used to make predictions about the population of 
relevant purchase requests. 

Specifically, the initial sample of 107 items found 21 items that were determined 
to be outside the scope of the audit. Based on these 21 items, the DODIG determined it 
li :ely that there was a amilar proportion of items in the entire population of 1430. The 
projection calculated 374 total population Hons outade the audit and thorefore the 
population was “adjusted” to 1056 items. A similar projection found SI 71 million of the 
original population value to be outside the audit’s scope. Therefore, the population value 
was reduced from $691.1 million to $520.1 million. Both of these projections were made 
the same way the projections were made for every other grouping used during the audit. 
This procedure is described in Chapter in.E. The results of this projection are then shown 
in Appendix D. Again, the only trxily valid way to adjust the population is to go item-by¬ 
item through the population and eliminate items outside the audit’s scope. Only when this 
process is complete can a usable sample be sdected. 

However, the DODIG did not actually use the “new” universe of 1056 items 
valued at $520.1 million. Instead the original universe size and value were used 
throughout the entire audit and in all calculations. The DODIG’s actual purpose of 
determining the “adjustment” was to ensure the total value of those items which were 
outside the audit’s scope was set aside and not mingled with any other part of the audit. 
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Therefore, the audit results did not become flawed as they may have foe "a^luated 
universe vahie” was actually used in calculatkms. 

2. Sample Siw 

As previously discussed, the chosen samfrfe size for tins audit was 107 out of 
1430 Hems. Deqnte the foct that this sudH of the Navy conunands was similar to nidits 
the DODIG conducted with the Air Force and Army, foere were no sanqde size sdectkm 
procedures available for review. Once the dedsimi to sample 107 hems was made, foere 
was no algorithm used to divide the 107 into the four strata (Interview with Dr. D. 
Barton, May 1994). Instead, consdous dedsimis were made to make 100% reviews of 
the top dollar value stratum, sample iq;>|»oxiinatdy 80% of the second stratum, and then 
divide the remaining hems over the bottom two strata. Once the decision was made to 
sanq>le percent from each stratum, the purchase requests were chosen by using simple 
random saiiq)liiig. 

3. Stratification 

One of the audhms’ first decisions after deciding to sample only those purchase 
requests valued at $100,000 or larger was to stratify the universe of purchase requests into 
four strata. As mentioned in Section B.2, the stated purpose of the strata sizing decisions 
was to ensure a review of a high percentage of the highest dollar value purchase requests. 
In other words, the top strata should not include too many hems. The strata breaks were 
based partially on “natural breaks” in the universe of purchase requests and partially on the 
level of supervisory review required for the purchase requests (Intoview with Mr. J. 
Chaney, May 1994). 
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One method to verify the effectiveness of stratification is to ctHnpare the 
standard deviation of the san^les before and after stratification. Secondly, the coefficient 
of variation diould be detennined to see the rdative variability. Equation (1) in Appendix 
A was used to make that calculation. As can be seen in Table X, simply dividing the 
sample between ASO and SPCC does not decrease the rdative variability - SPCC’s 
coefficient of variation (136.74) is higher than the combined coefficient of variation 
(128.71). The true advantage of stratification b apparait whoi the sample is broken imo 
strata by the size of the purchase request. Thb can be seen by the decrease in the 
coefficients of variation once the stratifications are made. 


Table X. SUMMARY OF STANDARD DEVIATIONS OF PURCHASE REQUEST 
VALUES AND COEFnCIENTS OF VARIATION 



Standard Deviation (S) 

Coefficient of Variation (Pet) 

ASO and SPCC 

2,759,542.57 

128.71 

ASO 

3.038,801.45 

121.45 

ASO - Stratum 1 

2,494,853.84 

25.56 

ASO - Stratum 2 

655,963.22 

20.45 

ASO - Stratum 3 

378,072.98 

25.98 

ASO - Stratum 4 

128,973.78 

54.80 

SPCC 

2,042,441.49 

136.74 

SPCC - Stratum 1 

2,525,926.84 

30.07 

SPCC - Stratum 2 

420,372.86 

13.02 

SPCC - ’atum3 

503,727.75 

32.51 

SPCC - um 4 

172,256.03 

67.72 


4. Confidence Level and Precuion 

An important statement in the report of audit findings was that “the sample 
results were projected with a 95% confidence level and a sampling precision of ± 3 
percent for dollars” (Jones, February 1, 1993, p. 33). In terms of the audit results, this 


translates to a $68,245,887 finding with a 95 percent confidence and a precision of less 
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than ± S2,047,377. In the audit wtKkmg papers the **niafBin oi error” is ttated as 
$552,396. Clearly, this is within the stated precision. However, dieee figures are analyzed 
in Section D to test their accuracy. 

C. SAMPLE SIZE ANALYSIS 
1. Precision 

Before attempting to verify the predsitm of ibt audit results, equations (2) 
through (4) in ^pendix A can be used to obtain a confidence interval for the projected 
number of hits made by the DODIG (sera in Appradbc M). First, it is necessary to obtain 
the population proportion. Secondly, an estimated variance riiould be determined. Then 
the variance can be used to obtain a confidence interval at the 95% confidence level. The 
step-by-step process is shown bdow. 

a. Estimator of the Population Proportion 

In order to estimate the proportion (fiaction) of purchase requests that were 
unnecessary, with projected monetary benefits, the DODIG auditors used equation (2) in 
Appendix A. The population was divided into strata and a sample was taken fi’om each 
stratum. The purchase requests were then thoroughly analyzed to see which were 
considered uimecessary with projected monetary benefits. The following calculations 
show how DODIG obtained the projected quantities for ASO and SPCC, respectively. 
These figures match those sera in ^pendix M. 

First, the formula for equation (2) is; 

” l>l 


Z8 




where N is the total universe size, Ni is the universe size by stratum as shown in Appendix 
B, and is the proportion of the sample that was found to be unnecessary with a 
projected monetary benefit. Next, values of these parameters were substituted imo the 
equation. 

ASO: p. - j^[8(0)+26(.15)+«6(.1765)+78<»(.25)] 


-.2394 


SPCC: p.~[2(0)+6(.l667) + 21(.3)+512(.35)] 

-^[186.5502] 

= .3447 


For example, ft=. 15 (above) for ASO and was the ratio 3/20 where 3 was the number of 
hits for ASO stratum 2 and 20 was the sample size for ASO stratum 2. The p^ values 
were computed fi'om the number of hits in ^pendix M and the stratum sample sizes in 
Appendix B. 

The .2394 and .3447 figures are used in Subsection c to determine the 
bounds on the errors of estimation. Also, the summations of N^p^ (212.797 and 186.55 
for ASO and SPCC, respectively) show how the DODIG projected the number of 
unnecessary purchase requests, listed in Appendix M. 
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b. Estinatcd Variaace 


The estimated variaiKe can be obtained by using equatkm (3) from Appendix 
A. The estimated variance is needed to olMain the confidence interval shown in 
Subsection c bdow. 




Calculating this solution is simplified by sq)ar8teiy calculating f ^ | |, which is 

V Nt 

also equal to nA)- VtPi)is determined as follows; 


^66-17^ 

■(.1765X.8235)] 

1 66 4 

i« J 

("789-24 

j(.25X.75)] 

1 789 

A 23 J • 


= 006744356; 


Substituting these values into equation (3) of ^pendix A gives: 

V(p^)= ^^^[O+(26)'(.00155)+(66)*(.00674)+(789)^.0079)] 


4950.955 

(889)* 


■006264486. 


Similarly for SPCC: 


V(p,)=0; 

V(A)-0; 
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and, therefore 

^Pm) - ^^[0+0+(2l)*(.012)+(512)*(.011506)] 


3021.608 

(541)* 


=■0103239. 


c. Bound on Error of Estinuition 

The results obtained from Subsections a and b are then used with equation 
(4) from .44>pendix A to obtain a 95% confidence levd. For ASO, the 95% confidence 
interval for the finction of purchase requests having projected monetary benefits is given 
by 

±2^y{pJ = .2394 ± 2V.006264486 
.23941.158297016 


or .0811^^^^.3977 


Similarly, for SPCC the 95% confidence interval is 

.344712V.010323899 

.34471.203213174 


or .1415^^^^.5479 

From Section 2 and Appendix M, the estimated number of unnecessary 
purchase requests for ASO and SPCC were 213 and 186, respectively. Using the results 
from this section, the 95% confidence interval for ASO would be from 72 (889 x .0811) to 
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354 (889 X .3977). For SPCC, the confidoKe interval is fitmi 77 to 296. The sixe of 
these intervals is quite large. “We could rechice this bound and nuke the estinutor more 
precise by imareasing the sample size” (Mendohall, 1990, p. 118). 

In an effort to get a tighter boumi on the errors of estimation, the strata for 
ASO and SPCC were combined and the calculations wen again made to determine a 9554 
confidence interval. The aggregate variance decreased to .00388 fit>m .00626 (for ASO) 
and .01032 (for SPCC). However, the range was still fiom 230 (1430 x .161) to 586 
(1430 X .41). The range is less than it is for ASO and SPCC calculated separately. 
However, the bounds on the number of estimated unnecessary purchase requests would 
still be considered large. 

2. Selecting Sample Size and Stratification 

The previous section looked at how much precision existed for the projected 
number of hits found by the DODIG. This section considers methods to increase the 
precision of sample hits along with guidelines to subdivide the chosen sample size among 
the given strata. The two options are based on allocating the sample size among the strata 
:n proportion to the universe size and then in proportion to the universe value, 
a. Sample Size Selection 

In order to ensure increased reliability fi’om sampling, methods have been 
used to select sample sizes based on (1) the expected rate of occurrence of the specified 
characteristic, (2) the desired confidence level, (3) the number of items in the universe, and 
(4) the desired reliability, .^pendix N was taken fi’om (Hill, 1962) and has taken into 
consideration each of these foctors. Its values are based on numerous calculations of 


32 








equation (S) in Appendix A. When the rate of occurrence is expected to be above 10%, 
the table with a 50% rate of occurroice is used to select a sample size. In this audit, the 
projected rate of occurrence, as given in .^pendix M, turned out to be 28% (399 1430). 

Finally, the sample size would have been 624 if the auditors were only looking at sample 
attributes and not sample values. 

b. Proportional Sample Sizes 

This section and section c assume that the sample size of 107 is correct and 
proposes two processes to spread the 107 over the four strata. In actuality, the sample 
size used by the DODIG was not proven correct. But, because the procedure to choose 
the 107 items was not available for review, this sample size will be the one allocated over 
the four strata. 


Table XL THE EFFECT OF PR( 

DPORTIO 

NALSAMl 

PLE SIZES 

(1) 

(2) 

(3) 

msm 

_ m _ 

(6) 

Stratum 

Universe 

Size 

Percent of 
Universe 

Sample 

Size 



ASO- 1 

8 

0.9 

8 

11.6 

1 

2 

26 

2.9 

20 

29.0 

2 

3 

66 

7.4 

17 

24.6 

5 

4 

789 


24 

34.8 

61 


889 

100.0 

69 

100.0 

69 

SPCC- 1 

2 

0.4 

2 

5.3 

1 

2 

6 

1.1 

6 

15.8 

1 

3 

21 

3.9 

10 

26.3 

1 

4 

512 

SwwBSyjaL&liSBaBaBgB 

20 

32.6 

35 


341 

100.0 

38 

100.0 

38 


One possibility is to spread the sample over the strata in proportion to the 
universe size (Cochran, 1963, p. 89). Table XI shows what percentage of the universe 
was in each stratum and then what percentage of the sample was in each stratum. Column 
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(3) shows that for both ASO and SPCC i^proximately 90% of the universe items are in 
the low dollar value category. Column (4) shows the sample sizes initially selected for 
each stratum. Colunm (S) shows the percentage that each stratum sample was of the 
whole for each ICP. If the sample size had stayed at 107 and was split between ASO and 
SPCC as in the audit and the sample had been spread over the strata in the same 
percentages as the four strata were of the universe, then the san 4 >le sizes would have been 
the spread seen in Colunm (6). However, this sample stratification would not have met 
the stated purpose of capturing a high percentage of the universe’s value with the smallest 
possible sample. 

c. Proportional Sample Values 

To reflect the goal of sampling a high percentage of the dollar value of the 
universe, the sample per stratum could have been determined by using the percentage of 
the universe value contained within each stratum (see Column (3) of Table XII). This 
method would avoid results like those currently seen in the fourth strata, where 
approximately S0% of the universe value is included but less than 10% of the sample 
value. When Column (3) is used to determine the appropriate sample size within each 
strata, the adjusted sample size (Colunm (6)) is not nearly as skewed to include the high 
dollar value strata as the actual sample itself (Colunm (7)). However, the top dollar value 
stratum sample sizes do not change for either ASO or SPCC. Instead, the two middle 
strata have smaller samples and the lowest dollar value strata sample sizes increase. 
Column (6) is calculated by multiplying the column (3) percentage by the original ICP 
total sample size. For example, ASO stratum 3 is calculated by multiplying 20.1% by 69 
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to obtain 14. When the product exceeds the entire stratum population as in SPCC stratum 
1 (8.6% X 38 = 3), then the difference (1) is added to the next highest stratum. This 
decision ensures the emphasis remains on the highest dollar value strata, as in the DODIG 
sampling plan. While this sample distribution is closer to the actual sanq)le than that seen 
in Section b, it still emphasizes the fourth stratum more than the actual cUd. 


Table XIL THE EFFECT OF PROPORTIONAL SAMPLE VALUES 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

■KJBH 

Stratum 

Univeise 

Percent of 
Universe 

Sample 

Percent of 
Sample 

Adjusted 

Sample 

Actual 

Sample 

ASO-1 

$78,092,994 

15.8 

$78,092,994 

45.2 

8 

8 



^1^9^^ 



14 

^20^ 

4 

234.191.420 


5.648.744 


33 

24 



100.0 

172.640.359 

100.0 

69 

69 

SPCC-1 


8.6 

16.801.000 

29.6 

2 

2 

2 

19.377.237 

9.9 

19.377.237 

34.1 

5 

6 

3 

34.665,847 

17.8 

15,493.707 

27.3 

7 

■BMH 

4 

124.441.089 

Ql 

5.087.281 


24 

20 


195.285.173 

100.0 


100.0 
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D. CONFIDENCE LEVEL AND PRECISION ANALYSIS 

When statements are made about a sample result’s confidence level and predsion, 
it is important to be able to test these measures. Using a procedure used by Des Raj in 
The Design of Sample Surveys, this section does exactly that. Equations (1), and (6) 
through (9), in Appendix A are used here to determine the precision of the stated results, 
given the stated 95% confidence level. Both SPCC and ASO are combined to ensure that 
the tightest possible interval is obtained. 

The first step is to gather the data shown in Table Xm. Columns (2) and (3) of 
Table XIII are taken from Appendix B. Column (4) can be calculated by dividing the total 
stratum hit value from Appendix M by the sample size (CoIumn(3)). For example, stratum 
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2 (S2I0,006) is calculated by dividii^ $5,460,151 by 26. Cohnmi (5) is calculated iraiQ 
equation (6) from Appendix A. When Tatde Xm is cmnplete, the populatkm’s UMal hit 
value is estimated. Using equation (7) from ^>pendbc A; 

L 

= 10(0) + 32(210,006) + 87(209,250) + 1301(32,441) 

is| 

= $67,130,830 


Table Xn 

IL DATA TO DETERMINE PRECISION 

WBsm 

'■am 

■m 

(4) 

(5) 

Stratum 

Universe 

(K) 

Sample 

("l) 

HhMean 

Hit Variance 
(^) 

1 

10 

10 


0 

2 

32 

26 



3 

87 

27 



4 


44 


WEmmm 


1430 

107 




This figure is less than the DODIG figure of $68,245,847 shown in ^pendix M. 
Secondly, equation (8) gives the mean value per purchase request of $46,945. Third, 
equation (9) gives the following: 

= 2,603,318,149,504 + 40,077,250,485,280 + 237,516,709,061,100 
= $280,197,277,695,884 

When this result is divided by the square of the population (1430), the variance of the 
estimate is determined to be $137,022,484 and the standard deviation $11,706. 

Using the procedures outlined in The Design of Sample Surveys, the next step is to 
determine the coefficient of variation using equation (1) in Appendix A; 
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= 24.935033% 

The coefiBdent of variation multiplied by the estimated total hit value of $67,130,830 
equals $16,739,094. This figure is then used in equation (10) of Appendix A: 

_ 16,739,094 
= $1,618,229 

Then, multiplying this figure by the 95% confidence level factor of 1.96 gives $3,171,729. 
Lastly, we dvide $3,171,729 by the estimated total hit value $67,130,830 and find an 
actual precision of ± 4.7247%. 

While 4.7247% may seem only slightly higher than 3%, in actuality 4.7Z47 is 
57.5% greater than 3%. Additionally, the range allowed by the auditors was rrom 
$66,198,510 to $70,293,264. However, the range calculated here was fi-om $63,959,100 
to $70,302,560. This range is over $2,000,000 greater than that claimed by the DODIG in 
the audit report. 
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E. DIFFERENCES 


1. Universe of S100,000purdiafcreqaciti 

While reviewing the background information for the data that was published in 
the final audit report, several discrepancies arose in the supportii^ data. The first 
category that dififm between the report and all supporting woildi^ papers is the dollar 
value of the univase of items which include purchase requests of $100,000 or more. 
While there was an initial figure for the entire universe value ($691.1 million), each of the 
four individual values was obtained through projections fix>m the sample of 107 items. 
Although the four sample values were r^nlarly being checked, re-checked, and changed 
during the audit, the four figures listed in Table XIV were ^patently never re-totaled after 
the projections were made to verify that they matched the original universe value. All 
four are listed in the working paper summary with the dollar figures seen in Table XTV. 
This $3 million difference is not significant but its impact carries forward to other 
categories, as seen in the next section. 


Table XIV. $100,00 

0 UNIVERSE BREAKDOWN 


Value 

Adjusted 

$170,972,701 

Undeterminable 

69,543,164 

Excessive 

156,563,299 

Reasonable 

297.062.229 

Total 

$694,141,393 


2. Adjusted Unhrene 

The second major difference discovered was the total dollar value of the adjusted 
universe. In all the major conclusions, there were two different dollar sizes for the 
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adjusted universe; $520.2 million, and $523.2 million. This difference in adjusted 
universe value ($3 million) is cterived from figures taken from the rq)oit. As seen in Table 
XrV, the universe value was $694,141,393. When the adjustment ($170,972,701) is 
subtracted, the adjusted universe value of $523,168,692 is obtained (see Colunm (1) of 
Table VII). Since the adjusted universe was not actually used in the audit estimate, this 
difference is of no practical significance. 

3. ASO Universe 

The next notable difference was the total dollar value for the 889 ASO purchase 
requests over $100,000. When projections were made for the three largest categories 
(undeterminable, reasonable, and excesnve), the working pqrer cover sheets correctly 
listed the ASO total as $495,886,170 (see Appendix B), however the detailed working 
papers, used to calculate the projections, totaled $498,856,171 (see column (4) of Table 
DC). 



Table XV. ADJUSTED UNIVERSE CORRECTIONS 


Original Value Corrected Value 


Undetermiiud)le 


Reasonable 


Excessive 




297,062,229 


156.563.299 

155.948.459 

$523,168,692. 

$521,102,679 



Table XVI. EXCESSIVE CORRECTIONS 
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Table Xm UNNECESSARY CORRECnWS 



Original Value 

Corrected Value 

Dtfferaaee 

Eumi 

$68,245,887 

$67,987,400 

v.'-v-. - 

Other 





$78,753,456 




The specific mistakes were a $30,000 shoitage fisr ASO stratum 2 and a 
$3,000,000 overage Sor ASO stratum 4. Tables XV throu^ XVn give the fectm^Kited 
projectitms for the cat^ories originally listed in Table Vn, Summary of Audit Results. 
While the origiiul errors appeared significant (a $2.97 million total difference), the true 
bottom line inqiact to be investigated is the change in the projected monetary benefits. As 
Table XVn shows, this amounted to $258,487 which is less than the average value of otM 
purchase request fi-om the original universe ($691,171,343 + 1430 $483,337) and only 

.38% (258,487 -i- 68,245,887) of the original projected monetary benefits. 

4. Excessive 

The firud notable difference that merits attention is the total value of excessive 
purchase requests. As Table Vn shows, ^^len the total for the adjusted universe is 
calculated, the dollar value for excessive purchase requests is $156,563,299. But i^dien 
the components of the excessive category are shown, they total $156,923,374. - a 
$360,075 difference. 

The best explanation for this diffoence is amilar to that described in Section 
E.l of this chapter. Again, it appears to be an instance ^^lere one set of figures was used 
to make the projection of the excessive value. Then, during a separate poim in the audit, 
the three components of the excessive value were projected. Apparently, the three 
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con^ponents were never totaled to ensure they womed to die le p OT t el y projected 
excessive total. 

5. Sumauuy 

During the review of all figures behind the audit report, it became dear that die 
figures eventually were d)tained in pieces. Retracing the i»ocess shows that die 
condusitHis obtained were baaed on shifting fixmdations. The moat imp(»tant figures are 
those used to make projections. As seen in Section E.3, $3 million and $30 thousand 
e^^ors are the basis for completely recalculating the projected monetary benefits of the 
entire audit. While the ultimate resuh may not be too much different, these corrections 
create a basis fisr questioning all assumptions made during the audit. 
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V. SUMMARY, CONCLUSIONS AND RECXNMMENDATIONS 


A. SUMMARY 

The objective of this tiiesU wu to examme how die Dqiartment of Defaue 
Inspector General (DODIG) tncoiporatod stratified —»npHne when uiditii^ inventcwy 
control prants. The primary fixus was on how the DODIG used stratified Mmpitng to 
project the potential monetary benefits of their finding it A secondary issue was to 
examine the stated confidence levd and predskm of their ccmchisUms to see if they were 
as tight as stated in the final rqwrt. 

Charter n introduced the methodology for using stratified sampling to make 
projections accurately. ¥or stratified samfding to be bes«ficial to the auditors, strata 
should be more homogeneous than the entire sanqile in order to reduce variability and 
reduce the total required sanqile size. Chapter n distinguished between frequency 
estimation and dollar value estimation, both of vdiich were used during the DODIG audit. 
The chapter also included a brief discussion of confidrace intervals and tlmir use in stating 
concluaons about a sample mean. The final section of Chapter n was a description of the 
term “stratification” in invemory rnanagonem. Stratification was critical for NAVSUP 
when reporting the impact of the audit. NAVSUP used three figures when calculating the 
audit’s impact. One of the figures, the DILS rate, was determined from reviewing the 
“STRAP’ while the audit was in process. 

The data that was coUected and used throughout the audit was described in 
Chapter m. Appendices B through M include the detailed data that was described in 
Chapter in. The beginning of Chapter m included a description of how the auditors 
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began the audh by narrowing the scope, choosing their san^ size and then divkhog their 
sample into the four strata. The following section briefly menticmed some of the ttidhore’ 
various considerations n^ien cat^orizii^ their findings. Charter m then fmsented the 
categorization of data that ultimately led to the DODIG’s most inqmrtant finding > the 
projected monetary benefits of unnecessary purchase requests ($68.2 ndllion). lunally, the 
chiq)ter concluded with NAVSUP’s use of the audit results to rqwrt the budgetary infract 
for the following fiscal year. 

Chapter IV presented an analy^ of the data coUected to itudce the audit 
concluaons. The first part of the chiqrter discussed the way the DODIG selected the 
“adjusted” universe, sample size and strata and their stated confidence level. The next 
section is the author’s analysis of the sanqrle size dedaons and the author’s investigation 
of alternate methods of selecting a sample size and stratifying the chosen sample. The 
chapter also presented and computed an ^gregate estimated confidence imerval based on 
the stratifications made during the audit and compared it to the stated accuracy of the 
DODIG report. The next section reviewed differing dollar figures used to make 
projections. In particular, this section explained how numbers changed as the audit 
progressed and how apparently no thorough final review was conducted to ensure that 
conclusions made early in the audit were based on the same figures as those used to make 
conclusions at the end of the audit. 

B. CONCLUSIONS 

During the DODIG audit, audited commands naturally put their emphasis on the 
individual findings. Each finding is closely scrutinized, verified, and argued over before 
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the auditors are allowed to rqx>rt their resuk. What is dearly more difficult to argue 
about are the iMojections made after the audit findiiigs are gathered. Even if the 
projections are explained, the precision of the findii^ is often beytmd sinqde 
interpretations. Therefore, the auditors’ repent results tend to be the final wmd. The 
precision of the audit projectiems are critical because the audited cemimands can then more 
accurately interpret the audit findings. 

During the process of reviewing the audit, results were found to be based on 
projections fix>m the chosen sample. However, one of the unusual condusions fit)m the 
sample was that the universe dze should be adjusted based on 21 sampled items valued at 
S41.1 million that were determined to be outside the scope of the audit. The DODIG 
auditors projected that 374 items totaling $171 million should be removed fi’om the 
original universe. The result was an unorthodox “adjusted universe” of 1056 items with 
an estimated value totaling SS20.2 million. However, all subsequem calculations and 
estimates were based on the original population size of 1430. Because an accurate 
universe is essential for all sampling, the original universe should have been completdy 
reviewed to remove items that were not to be induded in the audit and the resulting 
universe size used in all the subsequent analyses. 

Another finding of the audit review was that there was no documented procedure 
to explain why a sample size of 107 was chosen. Conversations with several of the 
DODIG statistidans revealed that no one could recall how a sample size of 107 was 
chosen. Despite the time between the audit and this thesis, 107 should have been 
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voifiable. Thi$ v^Mild have been posstUe if the procedure were documented, Hdrich it 
clearly was not. This issue is fiirther discussed in the reccmunendations. 

A significant finding of this thesis be found in A{^)codioes B and M. These 
q)pendices show that 63.7% of the potentuU monetary benefits of unnecessary purchase 
requests were derived fiiom the sanqrle of the fourth stratum. That sanqrle inchided only 
3.4% (44 -t- 1301) of the stratum universe of purchase requests and only 3.0% 
(10,736,025 + 358,632,509) of the stratum universe’s value. However, this same stratum 
included 91% of all purchase requests and nearly 52% of the purchase request value for 
the entire universe. In light of the very small sanqrle used in the fixirth stratum, the 
OODIG claim that their findings are based on a 95% confidence levd with a stated 3% 
level of precision is definitely opoi to question. 

A related conclusion of this thesis is that there was an arbitrary decision to divide 
the strata to ensure 100% inclusion of the highest dollar value purchase requests, 80% of 
the second highest strata, 30% of the next higher strata, and then pulling the remaining 
j„.aples fi’om the lowest dollar value strata. This focus on the dollar value of purchases 
created unrealistic assumptioru about the ruimber of line hems considered unnecessary. 
Specifically, the DODIG’s projected number of unnecessary purchase requests was 399 
(see i^pendix M). This conclusion assumes that neariy 28% (399 -i- 1430) of all ICP 
purchase requests were at least partially utmecessary. While this figure can be stated with 
95% confidence, the wide range (e.g., lack of precirion) of this interval must also be 
included when reporting such numbers. Instead, no precision was given for the number of 
utmecessary purchase requests. 
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A nu^or finding of this thesis resuhed from a test of the stated audit confidence 
level and precision of the dollar value estimates. Wlule the audit results were said to have 
a precision of ± 3% for the dollar vahm, the actual predsion determined was ± 4.7% or 
about S7% and $2 million larger than stated. This is inqxntant because all the auditors’ 
work was supposed to have led to a 95% confidence levd and ± 3% precision. The 
statistical accurate of the DODIG fimlings is ahical to anyone reviewing the rqrart. 
Without this accuracy, the findings are continually open to questions. 

C. RECOMMENDATIONS 

This thesis has highlighted several potential opportunities for improving the 
outcome of DODIG audits and the confidence in the findings rendered. 

1. DODIG Should Develop Standard Statistical Procedures and Publish 
Them 

During conversations with several of the DODIG representatives, it became 
clear that audits such as those conducted on the Navy ICPs will be occurring with 
increased frequency. Naturally, the audited commands are sensitive to findings that put 
them in a negative light. This creates an incentive for the audited commands to criticize 
the way the auditors make their findings and conclusions. In order to avoid other reviews 
like this thesis, and also to simplify the process for the auditors, standardized procedures 
of statistical steps should be developed and eventually published. This method of full 
disclosure on the part of the auditors would benefit everyone in the audit process. 
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2. A Suoxinct Sumauwy of the Aadit Fudiap Should Be Provided to 


Audited Commaads 

A thorough review of all the DODIG working papers and the final audit rqwrt 
clearly showed that, during the course of the audit, conclusions were readied and then 
changed because some of the early assumptions were changed. The resuh was that 
conclusions reached during various phases of the audit were based on differing 
assun^tions. For example, the projections made for the SPCC portion of the premature 
purchase requests (Appendix K) were ineiqilicably made uang an earlier resuh of an 
“excessive" value projection. However, throughout the rest of the audit, projections for 
all other groups were made using the original universe value. This small inconsistency 
created a difference in the two totals for “excessive” given in Columns (1) and (2) of 
Table vn. By requesting a summary of the conclusions, it would be easy for the audited 
commands to verify that all audit findings wo-e based on the same set of facts. 

3. Statistical Ranges Should Be Provided to Audited Commands With 

Supporting Data 

When commands are audited, they reedve the audh results with a statement 
that the auditors have 95% confidence in their findings within a certain range or precision. 
This is normally a given that is not open to question. However, this critical figure should 
be fully explained. When only 3% of one stratum is reviewed and the audit findings fi^om 
this stratum are used to extrapolate 63.7% of the findings, then it is essential to the 
audited commands that they have a complete understanding of why that was done. As 
mentioned earlier, the range of the audit findings are as important as the findings 
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themselves. Therefore, whoi a statement is made about confidence level and V* 
precision, these should be provided with the dollar values to which they refer. Then 
figures can be verified through the use of procedures like those used in Cht^ner fV.D. 
When this process is complete, a clear calculation of the precision is then available for 
review. 

4. Validate Sample Sizes After the Audit 

This thesis provides a process that can be used to determine the precision of 
selected sample sizes. For commands that are bong audited for attributes only (umike the 
ICPs), Chapter IV.C provides a process to verify the accuracy of the auditor’s precision. 
Additionally, commands being audited for both variables and attributes can also use the 
process to ensure the sample size is statistically valid. If the audited command concludes 
the auditor’s stated precision is inaccurate, the auditors should be approached immediately 
to allow them an opportunity to defend their findings or issue an addendum to the findings 
stated in the audit report. 

5. Additional Research is Needed 

The primary purpose of this thesis was to review the DODIG’s sampling plan 
which was used in the evaluation of the two ICPs’ purchase requests. What was not 
considered, but appeared to be an area of serious debate during the audit, were the issues 
of what constitute premature, unnecessary, or insufBcient purchases. A major factor in 
this determination was the definition of “stockage objective.” In order to narrow the sdjpe 
of this thesis, these terms were not debated. However, correspondence between 
NAVSUP and the DODIG clearly showed that the definitions were open to interpretation. 
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A iiK>re tn-dq>th look into this area would benefit the ICPs and others depoident oa 
stockage objectives to e£fectivdy numage warehouse inventCMies. 
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GLOSSARY 


Approved Acquisition O bjective (AAOV The quantity of an hem authoriased for 
peacetime a^ wartime requirements to equip ai^ sustain U.S. and Allied Forces in 
accordance with current DOD pdides and phms. 

Continyencv Retention Stock fCRSV That portion of the quantity of an hem greater than 
the AAO and economic retention stock tor whidi there is no inedictabie or 

quantifiable requirement, aiul that would normally be allocated as Potential 
Reutilization Stock , excq>t for a determination that tte quantity will be retained for 
specific contingendes. 

Due In Long Supp ly: Assets expected to arrive into the \^lesale supply system in excess 
of all known or expected requirements during some time period, usually thought of as 
in excess of the Retention Limh. 

Economic Retention Stock (ERSV That portion of the quantity of an hem greater than 
the AAO determined to be more economical to retain for fut^ peacetime issues than 
to dispose and satisfy projected future requirements through new procurement and/or 
repair. To warrant economic retention, an hem must have a reasonably predictable 
demand rate. 

Excessive : Purchase requests that fidl into other the “pronature” or “unnecessary” 
categories described below. 

Hardware Systems Command : A headquarters activity that is responsible for the 
procurement and technical support of weapons system requirements (e.g.. Naval Sea 
Systems Command and Naval Air Systems Command). 

Non-pro|ectable : A purchase request that could not be used to predict overall error 
because the ICPs had taken independent action to reduce occessive purchases. The 
actions wo'e taken because of ICP item manager or supervisory review or an HSC had 
directed a curtailment to the purchase. 

Potential Reutilization Stock : Material identified by an item numager for possible disposal 
but with potential for reutilization; or material that has the potential for being sent by 
an item manager to the Defense Reutilization and Marketing Service for possible 
reutilization by another Component or by a Federal, state, or local government agency. 

Premature : A purchase request with a quantity that exceeds the stockage objective by 
more than twelve months of forecasted requirements. 
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Reasonable : Purchase requests that do not fit into the “excessive” or “undeterminable” 
definitions described here. 




1 Limit : The maximum quantity of on-hand material that may be retained in 
stock. This quantity is deter.:an^ by summing AAO, ERS, and CRS. 


Safety Level : The value of the expected net inveotoiy (on hand minus backorders) just 
before an order arrives. This is a “cushion” of sto^ which is kept on hand to cover 
variations in demand during lead time. 


Undeterminable : A purchase request fisr whidi DODIG was unable to determine the 
reasonableness because the ICPs could not provide verifiable requirements data as of 
September 30,1990 or requirements were dependent on a critical management decision 
by a hardware systems command (HSC). 


Unnecessary : A purchase request with a quantity in excess of five years of forecasted 
requirements. 
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APPENDIX A: NOTATION AND EQUATIONS USED IN 
STRATIFIED SAMPLING 


Notations used in Stratified Sanqdiog: 

vahw of the “rth observation 
N total number of units in aU strata (the population) 
c - coeflBicieiit of variation 
s - standard deviation of a san^e 
T - the mean of the x, observations 

L - number of strata into which the population is divided for purposes of stratified 
sampling 

p = population proportion that possesses a specified charactoristic 

p^-dsi unbiased estimator of the pofnilation proportion p 

p^ - the population proportion in stratum “i” that possesses a specified charactoistic 

p = an unbiased estinutor of Pf 

Nf - total number of units in stratum “i” 

n = number of units sampled out of the population 

/i. = number of units sampled in stratum T’ 

V = estimated variance 

q=l.p 
= 1 - Pi 

i = stratum index 

= sample mean of the /th stratum 
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» estimate of population mean (F) 

E » standard error 

z »the normal deviate related to the confidence level (e.g., l.% for 95% confidence) 
mean estimator 

(1) coefficient of variation, a measure of rdative variability: (Neter, 1993, p. 84) 

c=.00(±) 


(2) estimator of the popuiation proportion p: (Mendenhall, 1990, p. 117) 

p.-jjp.p. 


(3) estimated variance of p„: (Mendenhall, 1990, p. 117) 




(4) bound on the error of estimation : (Mendenhall, 1990, p. 117) 




(5) standard error of a percent for a sample drawn fix>m a finite population: (EQll, 1962, 
p. A-l) 


±E 
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(6) vriaacc ci i saiqirie: (Nder, 1993. p. t2) 


±(x,-n 



M 

(8) estimated mean o f the pt^Hilation; (Rig, 1972, p. S3) 

^ tel 


(9) variance of the esdmator x : (Rig, 1972, p. S3) 
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APPENDIX B: SUMMAflY OF UNtVEfISE AND SAMPLE VALUES 
































































































C: COMPtITg UST OF SAMPtEP ITEMS CINITIALLY) 


Numbart 


1|NA1<01 


2 NA1-02 


3 NA1-03 


NA1-04 


5 NA1-05 


6 NA1-06 


NA1-07 


NA1-08 


NA2-01 


10 NA2-02 


11 NA2-03 


12 NA2-04 


13 NA2-0S 


14 NA2-06 


15 NA2-07 


16 NA2-08 


17 NA2-09 


18 NA2-10 


19 riA2-11 


20 NA2-12 


21 NA2-13 


22 NA2-14 


23 NA2-15 


24NA2-16 


25 NA2>17 


26 NA2>18 


27 NA2-19 


28 NA2-20 


29 

NA3-01 

30 

NA3-02 

31 

NA3-03 

32 

NA3-04 

33 

NA3-05 

34 

NA3-06 

35 

NA3-07 

36 

NA3-08 

37 

NA3-09 

38 

NA3-10 

39 

NA3-11 

40 

NA3-12 

41 

NA3-13 

42 

NA3-14 

43 

NA3-15 

44 

NA3-16 

45 

NA3-17 


Extandad Vahia 


413.666.416 


11,967,280 


11,104,740 




9,006,800 


7.935,475 



2.902.650 


3,480,000 


3,112,110 


3,893,507 


3,035,373 


2,774,636 


2,602,380 


2,629,424 


4,916,280 


2,701,218 


3,141,600 


3,655,000 


2,569,869 


3,170,952 


3,272,160 


2.549,261 


2,827,366 


3,228,039 


3,007,680 


4,690,796 


1,486,694 


1,241,267 


1,314,587 


1,269,709 


1,821,430 


1,673,360 


1,122,028 


2,340,030 


1,346,856 


1,141,950 


1,067,950 


1,146,958 


1,334,520 


1,159,297 


1,994,278 


1,254,300 


2,023,104 



24,738,318 

















































































































































APPENDIX C: COMPLETE UST OP 



ITEMS (mmAUY) 


NA4>01 


47 NA4>02 


48 NA4-03 


49 NA4-04 


50 NA4-0S 


51 NA4-06 


52 NA4-07 


53 NA4-08 


54 NA4-09 


NA4-10 


56 NA4-11 


57 NA4-12 


NA4-13 


59 NA4-14 


60 NA4-15 


61 NA4-16 


62 NA4-17 


NA4-18 


64 NA4-19 


65 NA4-20 


NA4-21 


67 NA4-22 


68 NA4-23 


NA4-24 



70 

NS1-01 

71 

NS1-02 


72 

NS2-01 

73 

NS2-02 

74 

NS2-03 

75 

NS2-04 

76 

NS2-05 

77 

NS2-06 


78 

NS3-01 

79 

NS3-02 

80 

NS3-03 

81 

NS3-04 

82 

NS3-05 

83 

NS3-06 

84 

NS3-07 

85 

NS3-08 

86 

NS3-09 

87 

NS3-10 




NS4-01 

89 

NS4-02 

90 

NS4-03 


411.350 


126,943 


587.070 


298,580 


163.896 




180,000 


225,146 


117,539 


128.370 


227,125 


294,000 


434,968 


113,920 


313,200 


121,726 


113,971 


185,786 


340,000 


438,070 


106,587 


130,314 


186,480 


203,274 


10,186,600 


6,614,400 


3,840,000 


3,592,000 


3,150,000 


3,084,156 


3,045,690 


2,665,391 


1,750,000 


1,602,685 


1,108,500 


1,150,000 


1,152,000 


1 . 102.000 


2,232,510 


2 , 100,000 


2,256,012 


1,040,000 


225,215 


350,980 


145,000 
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APPENDIX C: COMPUETE U8T OP 8AMPU0 ITBy» (MmAUVI 


91 

NS4^ 



214,773 


92 

NS4-05 



155,682 


EE 

NS4-06 



109,152 



NS4-07 





EE 

NS4-08 



610,977 


EE 

NS4-09 



266,489 



NS4-10 



144,750 


ee 

NS4-11 



268,935 


99 

NS4-12 



170,880 


iEE 

NS4-13 



105,934 



NS4-14 



125,844 



NS4-15 



291,816 


103 

NS4-16 



731,766 


104 

NS4-17 



278,694 



NS4-18 



129,422 


fEU 

NS4-19 



153,920 


107 

NS4-20 



125,022 







5,087.281 






$229,399,582 


* A = ASO 





S > SPCC 





Ist Number » Stratum 




2nd and 3rd Numbers « 

Mumbar Within Stratum I 


58 











































































































































APPENDIX E: COMPLETE UST OF SAMPLED ITEMS (REVISED) 


Samoie Numbers I National Stock Numbers 


EQI 


m 

\m 

\m 

\m 


R3 

\m 

jcTH 


lES 
I EE] 


lES 

m 

S 

lEEll 

\m 


NA1-01 


NA1-02 


NA1-03 


NA1-04 


NA1-06 


NA1-07 


NA1-08 


NA2-01 


NA2-02 


NA2-04 


NA2-05 


NA2-06 


NA2-07 


NA2-08 


NA2-10 


NA2-14 


NA2-15 


NA2-16 


NA2-17 


NA2-18 


NA2-19 


NA2-20 


NA3-01 


NA3-02 


NA3-03 


NA3-04 


NA3-05 


NA3-06 


NA3-07 


NA3-08 


NA3-10 


NA3-11 


NA3-12 


NA3-13 


NA3-14 


NA3-15 


NA3-17 


NA4-01 


NA4-02 


NA4-03 


NA4-05 


NA4-07 


NA4-08 


NA4-10 





1615-01-201-9639 


f660-01-277-8238 


2840-01-251-7227 


5960-LL-NEE-A190 


2840-LL-WY5-5024 


6605-01-054-3776 


1615-01-201-9601 


1615-01-158-9678 


1650-01-161-4420 


2840-01-142-8818 


1560-01-155-7014 


5998-01-306-1972 


4920-01-124-9246 


4920-01-220-4520 


5865-01-196-9869 


1430-01-325-2512 


4920-01-279-8220 


5855-01-052-6849 


1620-00-761-4903 


4920-01-156-1393 


1270-01-256-8264 


4920-01-124-9245 


1620-01 


5895-01 


1560-01 


1740-01 


5841-01 


2835-01 


2925-01 


1630-01 


1730-01 


6720-01 


7021-01 


6605-01 


6605-01 


5963-01 


1280-01 


-177-1891 


-303-7755 


-300-7768 


-062-1657 


-004-7531 


-256-8378 


-277-3508 


-106-4900 


-126-6239 


-181-5872 


-283-3749 


-245-8209 


-027-4172 


-154-2794 


-095-2982 


6610-01-278 


1560-01-284 


6615-01-183 


5999-01-271 


4920-01-251 


1610-00-887 


2840-01-150 




9291 


5093 


7413 


1243 


7174 


0392 


6734 




Quanti 


94 


74 


51 


253 


16 


61 


Extendad Value 


$13,666,416 


11,967,280 


11,104,740 


10,634,238 


7,935,474 


7,381,000 


6,397,046 


2,902,650 


3,480,000 


3,893,507 


3,035,373 


2,774,636 


2,602,380 


2,629,424 


2,701,218 


3,170,952 


3,272,160 


2,549,261 


2,827,366 


3,228,039 


3,007,680 


4,690,796 


1,486,694 


1,241,267 


1,314,587 


1,269,709 


1,821,430 


1,673,360 


1,122,028 


2,340,030 


1,141,950 


1,067,950 


1,146,958 


1,334,520 


1,159,297 


1,994,278 


2,023,104 


411,350 


126,943 


587,070 


163,895 


180,000 


225,146 


128,370 


227,125 


Subtotals 


$69,086,194 


46,765,442 


22,137,162 
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APPENDIX E: COMPLETE UST OF SAMPLED ITEMS (REVISED) 


\m 


NA4-12 


NA4-15 


NA4-16 


NA4-18 


NA4-19 


NA4-21 


NA4-22 


NA4-23 


NA4-24 



NS2-01 


NS2-02 


NS2-03 


NS2-04 


NS2-05 


NS2-06 


NS3-01 


NS3-02 


NS3-03 


NS3-04 


NS3-05 


NS3-06 


NS3-07 


NS3-08 


NS3-09 


NS3-10 


NS4-01 


NS4-02 


NS4-03 


NS4-04 


NS4-06 


NS4-07 


NS4-08 


NS4-10 


NS4-12 


NS4-13 


NS4-14 


NS4-15 


NS4-17 


NS4-19 


NS4-20 


4810-01-271-8852 


6615-01-129-8410 


1280-01-186-1434 


1680-01-242-9698 


2840-01-281-3618 


7050-01-098-5523 


1680-01-159-9153 


6680-01-175-9116 


1620-01-158-5958 



5895-01 


2010-01 


5960-01 


6625-01 


6625-01 


6625-01 


-281-2401 


-111-9593 


-302-4456 


-234-0485 


-146-1554 


-233-7104 


6150-01-306-7242 


1420-01-108-5915 


5820-01-020-2762 


4310-01-187-5041 


5999-00-619-7838 


4320-01-220-1747 


6625-01-258-3140 


5845-01-307-6466 


6625-01-259-7355 


6695-01-299-8473 


6130-01-155-2338 


5820-00-334-8407 


5999-01-255-1816 


2825-00-371-7899 


6605-01-030-0004 


2010-01-222-5283 


5985-01-119-3998 


4820-01-090-6529 


5915-00-527-9524 


4320-01-062-1473 


2010-01-144-2462 


6625-01-268-6800 


2865-01-164-1509 


2990-01-134-6899 


5998-01-183-7818 


S = SPCC 


1st Number Stratum 


2nd and 3rd Numbers = Number Within Stratum 





294.000 


313,200 


121.726 


185,786 


340,000 


106,587 


130,314 


186,480 


203,274 


3,840,000 


5,388,000 


3,150,000 


3,084,156 


3,045,690 


2,665,391 


1,750,000 


1,602,685 


1,108,500 


1,150,000 


1,152,000 


1 , 102,000 


2,232,510 


2,100,000 


2,256,012 


1,040,000 


225,215 


350,980 


110,000 


214,773 


109,152 


482,030 


226,288 


144,750 


170,880 


105,934 


125,844 


291,816 


278,694 


153,921 


125,022 


3,931,266 


mLESMl 


21,173,237 


15,493,707 




3,115,299 


$188,316,707 














































































































































































APPENDIX F: SUMMARY OF ADJUSTED UNIVERSE AND SAMPLE VALUES 



62 

















































































lASO 


Stratum 

I_^ 

_ 2 _ 

_3_ 

4 


ISPCC 


Stratum 

_ 1 _ 

_ 2 _ 

_3_ 

4 


[TOTAL 


Stratum 

1 

2 

3 

4 

















































































APPENDIX H: REASONABLE PURCHASE REQUESTS 


lASO 

_ 



_ 




..~""l 

■■■■ 



IHIHii 

hhbi^h 





Stratum 


Hits 


HitVakia 


Projected 

Hits 


Projected Value 

1 


3 


♦31,007,495 


3 


$31,007,495 

2 


6 


25,869,400 


8 


33,814,662 

3 


5 


10,827,385 


19 


43,638,953 

4 


5 


2.376,480 


164 


99.788.676 



19 


$70,080,760 


195 


$208,249,786 










SPCC 




















Hits 


Hit Value 


Projected 

Hits 


Projected Velue 

1 


1 


$614,400 


1 


$6,614,400 

2 


5 


17,717,238 


5 


17,717.238 

3 


6 


11,560,810 


13 


25.866,326 

4 


7 


1.578.600 


179 


38,614.479 



19 


$31,471,048 


198 


$88,812,443 



















TOTAL 



_ 













IHHIH 


Stratum 


Hits 


Hit Value 


Projected 

Hits 


Projected Value 

1 


4 


$31,621,895 


4 


$37,621,895 

2 


11 


43,586,638 


13 


51,531,900 

3 


11 


22,388,195 


32 


69,505,279 

4 


12 


$71,659,360 


343 


138.403.155 



38 


$169,256,088 


392 
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APPENDIX I: EXCESSIVE PURCHASE REQUESTS 


A80 


Stratum 

1 

2 

3 

4 



spec 


Stratum 

1 

2 

3 

4 




TOTAL 


Stratum 

1 

2 

3 

4 













Hits 




Projected 

Hits 


Projected Value 


2 


«20,063,461 


2 


$20,063,461 


6 


11,778,245 


8 


14,334,254 


11 


10,596,389 


35 


35,656,225 


18 


2,701,732 


534 


86,509,359 


37 


$45,139,827 I 


579 
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APPENDIX J: NON-PROJECTABU PURCHASE REQUESTS 

















































































APPENDIX K: PREMATURE 


REQUESTS 
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APfENOIXL: UNNECESSAIIY PURCHASE RfQUESTS 

























































































































































UNKNOWN 180%!. CONPIOOIC8 LfVEL 98% 








± 2% 

± 3% 

± 4% 

± 5% 

200 






300 






400 





196 

500 





217 

1.000 



.. 

375 

278 

1,500 




429 

306 

2,000 



mtmm 

462 

322 

2,500 


1225 

748 

484 

333 

3,000 


1334 

787 

500 

341 

3,500 


1424 

818 

512 

346 

4,000 


1500 

842 

522 

350 

4,500 


1566 

863 

530 

354 

5.000 


1622 

879 

536 

357 

6,000 


1715 

906 

546 

361 

7,000 


1788 

926 

553 

364 

8,000 


1847 

942 

558 

367 

9,000 


1895 

954 

563 

368 

10,000 


1936 

964 

566 

370 

15,000 


2070 

996 

577 

375 

20,000 


2144 

1013 

583 

377 

25,000 


2191 

1023 

586 

378 

50,000 


2291 

1045 

593 

381 

100,000 


2345 

1056 

597 

383 

Note: This ta 

bie should be used only when the auditor is unabi 

e or 


unwilling to fix a maximum occurrence rate to be expected. This 


conservative approach will result in a much larger sample size than 
will be found in tables where an expected maximum rate is estimated. 
Source; Sampling in Auditing, Hill,1962. p. A-1 | _ 
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